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DETAILED ACTION 

Specification 

1 . The abstract of the disclosure is objected to because it includes improper language such as 
"disclosed" (in line 1.) See MPEP 608.01(b). 

Corrections are required. 

Claim Objections 

2. Claim 2 is objected to because of the following informalities: 
In claim 2, line 2: "and is" should be --is--. 

Correction is required. 

Claim Rejections - 35 USC § 103 

3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness 
rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 



4. 



Claims 1-3 and 6 are rejected under 35 U.S.C. 103(a) as being unpatentable over Barber et al 
(U.S. patent No. 5,579,471) in view of Rhoads (US. patent No. 6,084,227.) 
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As to claim 1, Barber et al teaches a multimedia data structure (see column 6, lines 50-53) 
reflecting change of a user relevance feedback for determining weights (see column 8, lines 17- 
21, and see column 9, line 65 through column 10, line 2) of image features used for an image 
search (see column 3, lines 58-60), comprising: 

(a) information describing the features of a certain image (see column 5, lines 30-35); 
(d) wherein the weights of the image features are determined (see column 8, lines 15-21, 

where "determining the weight" is read on "image characteristic is weighed"), and the image 
features according to the determined weights are used for the image search (see column 17, line 
64 through column 18, line 2.) 
Barber et al does not teach: 

(b) recent user feedback information when the weights of the features of the certain image 
are learned and determined by the user relevance feedback; 

(c) whole feedback information when the weights of the features of the certain image are 
learned and determined by the user relevance feedback; and 

(d) the weights are determined considering both the recent user feedback information and the 
whole feedback information. 

Rhoads teaches an imaging system (see Abstract), in which he teaches: 
(b) recent user feedback information when the weights of the features of the certain image 
are learned and determined by the user relevance feedback (see column 35, lines 29-39); (c) 
whole feedback information when the weights of the features of the certain image are learned 
and determined by the user relevance feedback (see column 35, lines 40-50, where "whole 
feedback" is read on "weighted average"); and (d) the weights are determined considering both 
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the recent user feedback information and the whole feedback information (see column 35, lines 
50-54, and see column 71, lines 51-64.) 

Therefore, it would have been obvious to a person having ordinary skill in the art at the time 
the invention was made to have modified Barber et al to include recent user feedback 
information when the weights of the features of the certain image are learned and determined by 
the user relevance feedback; whole feedback information when the weights of the features of the 
certain image are learned and determined by the user relevance feedback; and, the weights are 
determined considering both the recent user feedback information and the whole feedback 
information. 

It would have been obvious to a person having ordinary skill in the art at the time the 
invention was made to have modified Barber et al by the teachings of Rhoads because including 
recent user feedback information when the weights of the features of the certain image are 
learned and determined by the user relevance feedback; whole feedback information when the 
weights of the features of the certain image are learned and determined by the user relevance 
feedback; and, having the weights determined considering both the recent user feedback 
information and the whole feedback information, would allow the system to keep current and up- 
to-date characteristics (weights) of images based on individual users' feedback and by combining 
the individual feedback into whole feedback for a given image, and use the whole feedback for 
searching for a particular image in the image database. 
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As to claim 2, Barber et al as modified teaches wherein the recent feedback information is 
determined for a predetermined time period or by a predetermined frequency (see Rhoads , 
column 33, lines 49-58.) 

As to claim 3, Barber et al as modified teaches wherein the recent feedback information is a 
weight value learned by the recent user feedback or a similar image information (see Rhoads, 
column 35, lines 33-40), and the whole feedback information is represented by a weight value 
learned by all the feedback given till now (see Rhoads , column 35, lines 39-50.) 

As to claim 6, Barber et al as modified teaches the method comprising: 
representing the recent user feedback information by a similar image list (see Barber et al , 
column 5, lines 35-42); and 

reflecting a recent user feedback pattern (see Barber et al , column 12, lines 38-44) by the 
similar image, list using a queue algorithm (see Barber et al column 6, lines 37-50, where 
"queue algorithm" is read on "thumbnail data representation", also see column 9, line 65 through 
column 10, line 2, where "queuing" is read on "the order of the images returned".) 

5. Claim 4 is rejected under 35 U.S.C. 103(a) as being unpatentable over Barber et al (U.S. 
patent No. 5,579,471) in view of Rhoads (U.S. patent No. 6,084,227) as applied to claims 
1-3, and 6 above, and further in view of Blanchester (U.S. Patent No. 6,141,438.) 
As to claim 4, Barber et al as modified does not teach the data structure further comprising 

recent user feedback reliability information representing how reliable the recent user feedback 
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information is, and whole feedback reliability information representing how reliable the whole 
feedback information is. 

Blanchester teaches a document authentication control device (see Abstract), in which he 
teaches the data structure further comprising recent user feedback reliability information 
representing how reliable the recent user feedback information is, and whole feedback reliability 
information representing how reliable the whole feedback information is (see column 1, lines 64- 
67, and see column 3, line 64 through column 4, line 52.) 

Therefore, it would have been obvious to a person having ordinary skill in the art at the time 
the invention was made to have modified Barber et al as modified to include the data structure 
further comprising recent user feedback reliability information representing how reliable the 
recent user feedback information is, and whole feedback reliability information representing how 
reliable the whole feedback information is. 

It would have been obvious to a person having ordinary skill in the art at the time the 
invention was made to have modified Barber et al as modified, by the teaching of Blanchester , 
because including the data structure further comprising recent user feedback reliability 
information representing how reliable the recent user feedback information is, and whole 
feedback reliability information representing how reliable the whole feedback information is, 
would result in more accurate image searches in an image database. 
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6. Claims 1 1 and 12 are rejected under 35 U.S.C. 103(a) as being unpatentable over Barber et al 
(U.S. Patent No. 5,579,471) in view of Rhoads (U.S. Patent No. 6,084,227) as applied to 
claims 1-3 and 6 above, and further in view of Evelyn- Veere et al (U.S. Patent No. 
4,176,395.) 

As to claim 1 1, Barber et al as modified teaches wherein the recent user feedback 
information is represented as a similar image list (see Barber et al column 5, lines 35-38), and 
the similar image list has an image list structure composed of a similar image identification (see 
Barber et al column 5, lines 38-42), a score reflecting the current feedback (see Barber et al 
column 14, lines 46-55.) 

Barber et al as modified does not teach a waiting duration representing a time period between 
the final feedback time point and the present time point. 

Evelyn-Veere et al teaches an interactive irrigation control system (see Abstract), in which 
she teaches a waiting duration representing a time period between the final feedback time point 
and the present time point (see column 34, lines 41-47, and see column 35, lines 53-65.) 

Therefore, it would have been obvious to a person having ordinary skill in the art at the time 
the invention was made to have modified Barber et al as modified to include a waiting duration 
representing a time period between the final feedback time point and the present time point. 

It would have been obvious to a person having ordinary skill in the art at the time the 
invention was made to have modified Barber et al as modified, by the teaching of Evelyn- Veere 
et al because having a waiting duration representing a time period between the final feedback 
time point and the present time point will indicate to the user how recent the feedback on a given 
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image is and would further indicate whether the current user's feedback is received and/or 
implemented. 

As to claim 12, Barber et al as modified teaches the image list managing method comprising: 
managing a similar image list in a manner that only images whose scores are over a 
predetermined threshold number are maintained in the list (see Barber et ah column 14, lines 46- 
64) or only images whose scores are N on upper positions of the list if a size of the list is N (see 
Barber et al column 14, lines 65-67, where "ranking the image" and "best scores" are 
discussed.) 

7. Claims 14-16 are rejected under 35 U.S.C. 103(a) as being unpatentable over Barber et al 
(U.S. patent No. 5,579,471) in view of Rhoads (U.S. patent No. 6,084,227), and further in 
view of Blanchester (U.S. Patent No. 6,141,438.) 

As to claim 14, Barber et al teaches a method of determining weights of image features (see 
column 8, lines 17-21, and see column 9, line 65 through column 10, line 2) in a system for 
determining the weights of the features used for an image search (see column 8, lines 15-21, 
where "determining the weight" is read on "image characteristic is weighed) by the user 
relevance feedback, the method comprising the steps of: 

(a) providing a multimedia data structure including information describing the features of a 
certain image (see column 6, lines 50-53), 

(c) determining the weights of the image features (see column 8, lines 15-21, where 
"determining the weight" is read on "image characteristic is weighed.) 
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Barber et al does not teach: recent and whole feedback information; updating the feedback 
information by learning them in response to the user feedback. 

Rhoads teaches an imaging system (see Abstract), in which he teaches: recent and whole 
feedback information; and updating the feedback information by learning them in response to the 
user feedback (see column 35, lines 29-39, and see column 35, lines 40-50, where "whole 
feedback" is read on "weighted average".) 

Therefore, it would have been obvious to a person having ordinary skill in the art at the time 
the invention was made to have modified Barber et al to include recent and whole feedback 
information; and updating the feedback information by learning them in response to the user 
feedback. 

It would have been obvious to a person having ordinary skill in the art at the time the 
invention was made to have modified Barber et al by the teachings of Rhoads because including 
recent and whole feedback information; and updating the feedback information by learning them 
in response to the user feedback, would allow the system to keep current and up-to-date 
characteristics (weights) of images based on individual users' feedback and by combining the 
individual feedback into whole feedback for a given image, and use the whole feedback for 
searching for a particular image in the image database. 

Barber et al as modified still does not teach: reliability information of the respective 
feedback information; and updating their reliabilities by learning them in response to the user 
feedback. 

Blanchester teaches a document authentication control device (see Abstract), in which he 
teaches reliability information of the respective feedback information; and updating their 
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reliabilities by learning them in response to the user feedback (see column 1, lines 64-67, and see 
column 3, line 64 through column 4, line 52.) 

Therefore, it would have been obvious to a person having ordinary skill in the art at the time 
the invention was made to have modified Barber et al as modified to include reliability 
information of the respective feedback information; and updating their reliabilities by learning 
them in response to the user feedback. 

It would have been obvious to a person having ordinary skill in the art at the time the 
invention was made to have modified Barber et al as modified, by the teaching of Blanchester . 
because including reliability information of the respective feedback information; and updating 
their reliabilities by learning them in response to the user feedback, would result in more 
accurate image searches in an image database. 

As to claim 15, Barber et al as modified teaches wherein the recent feedback information is 
represented by a weight value learned by the recent user feedback or a similar image information 
(see Rhoads, column 35, lines 33-40), and the whole feedback information is represented by a 
weight value learned by all the feedback given till now (see Rhoads , column 35, lines 39-50.) 

As to claim 16, Barber et al as modified teaches wherein the reliability of the recent user 
feedback information is determined, in proportion to a consistency of a recently used pattern or 
feedback (see Blanchester , column 4, lines 52.) 
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Allowable Subject Matter 

8. Claims 5 and 7-10, 13, and 17 are objected to as being dependent upon a rejected base claim, 
but would be allowable if rewritten in independent form including all of the limitations of the 
base claim and any intervening claims. 

9. The following is a statement of reasons for the indication of allowable subject matter: 

The prior art of record, Barber et al (U.S. Patent No. 5,579,471), Rhoads (U.S. Patent No. 
6,084,227), Blanchester (U.S. Patent No. 6,141,438), and Evelyn- Veere et al (U.S. Patent No. 
4,176,395) do not disclose, teach, or suggest the claimed limitations of (in combination with all 
other features in the claims) : r \ -in 



wherein the recent user feedback reliability information is expressed by j r — 

where, N is the number of feedback, m is the number of images in the similar image list, and ni 
is the number of feedback given to the i-th image, as claimed in claim 5. 

The prior art of record, Barber et al (U.S. Patent No. 5,579,471), Rhoads (U.S. Patent No. 
6,084,227), Blanchester (U.S. Patent No. 6,141,438), and Evelyn- Veere et al (U.S. Patent No. 
4,176,395) do not disclose, teach, or suggest the claimed limitations of (in combination with all 
other features in the claims): 

wherein the queue algorithm comprises the steps of: 

(a) checking whether a similar multimedia object exists in a current queue when the 
corresponding object is fed back ; 
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(b) if it is checked that the corresponding object does not exist in the current queue, inputting 
the corresponding object to an uppermost space of a queue entrance, setting the number of 
feedback of the corresponding object to "1", and shifting objects existing in the queue to lower 
positions by one space; 

(c) if it is checked that the corresponding object exists in the current queue, increasing the 
number of feedback of the corresponding object, and shifting the objects existing in the queue to 
upper positions by "N"; and 

(d) if any object is shifted to the lower position over a size of the queue at the respective 
steps, deleting the corresponding object from the queue, as claimed in claim 7. 

Claims 8-10 are objected to as being dependent upon the objected to dependent claim 7. 

The prior art of record, Barber et al (U.S. Patent No. 5,579,471), Rhoads (U.S. Patent No. 
6,084,227), Blanchester (U.S. Patent No. 6,141,438), and Evelyn- Veere et al (U.S. Patent No. 
4,176,395) do not disclose, teach, or suggest the claimed limitations of (in combination with all 
other features in the claims): 

A feedback reflecting score updating method for the multimedia data structure reflecting the 
recent feedback, comprising updating image scores in a mariner that: 

(a) with respect to images fed back from similar images, a following calculation is 
performed; and 

Score (new) = Score (current) X fW (Waiting Duration) + 1 

(b) with respect to other images not fed back, a following calculation is performed; 
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Score(new) = Score (current) XfW(Waiting Duration) 
(c) wherein fW(Waiting Duration) is a function having the characteristic that it returns 
smaller value as the waiting duration becomes larger, as claimed in claim 13. 

The prior art of record, Barber et al (U.S. Patent No. 5,579,471), Rhoads (U.S. Patent No. 
6,084,227), Blanchester (U.S. Patent No. 6,141,438), and Evelyn- Veere et al (U.S. Patent No. 
4, 176,395) do not disclose, teach, or suggest the claimed limitations of (in combination with all 
other features in the claims): 

wherein the reliability of the whole feedback information is determined in proportion to the 
number of feedback concerned in learning, as claimed in claim 17. 

Conclusion 

10. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 

The following patents are cited to further show the state of art with respect to methods 
and systems of searching images in an image database, providing feedback on the image 
characteristics, and measuring feedback reliability in general: 

U.S. Patent No. 5,748,231 to Park et al . 

U.S. Patent No. 6,292,575 to Bortolussi et al . 

U.S. Patent No. 5,281,995 to Terashita et al . 

U.S. Patent No. 5,652,899 to Mays et al . 
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1 1 . Any inquiries concerning this communication or earlier communications from the examiner 
should be directed to Tony Mahmoudi whose telephone number is (703) 305-4887. The 
examiner can normally be reached on Mondays-Fridays from 08:00 am to 04:30 pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Dov Popovici, can be reached at (703) 305-3830. 

tm 

August 08, 2002 
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SUPERVISORY PATENT EXAMINER 
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